Method for quantitative video-microscopy and associated system and computer software program product

ABSTRACT

A method of determining an amount of at least one molecular species in a sample from an image of the sample captured by an image acquisition device is provided, each molecular species being indicated by a dye. A dye space representation of a plurality of dyes is formed by orthogonally adding the correspondence tables of the dyes, each correspondence table having a plurality of normalized RGB triplets and incrementally extending from 0% to 100% transmittance. The dye space representation has one dimension for each dye and provides a reference model for a combination of the plurality of dyes. Each pixel of an image of the sample stained with the combination of the plurality of dyes is compared to the reference model, each pixel having a color defined by an RGB triplet, so as to determine an optimal combination of normalized RGB triplets from the respective correspondence tables of the dyes producing the color of the respective pixel. An artificial image of the sample is then formed from the normalized RGB triplets for each dye as determined from the optimal combination. The artificial image thereby indicates a distribution of the respective dye over the sample image and facilitates determination of the amount of the corresponding molecular species. Associated methods, systems, and computer software program products are also provided.

FIELD OF THE INVENTION

The present invention relates to image analysis and, more particularly,to a method for quantitative video-microscopy in cellular biology andpathology applications and an associated system and computer softwareprogram product therefor.

BACKGROUND OF THE INVENTION

Effective analysis of microscopic images is essential in cellularbiology and pathology, particularly for detection and quantification ofgenetic materials such as, for example, genes or messenger RNA, or theexpression of this genetic information in the form of proteins such as,for example, through gene amplification, gene deletion, gene mutation,messenger RNA molecule quantification, or protein expression analyses.Gene amplification is the presence of too many copies of the same genein one cell, wherein a cell usually contains two copies, otherwise knownas alleles, of the same gene. Gene deletion indicates that less than twocopies of a gene can be found in a cell. Gene mutation indicates thepresence of incomplete or non-functional genes. Messenger RNAs (mRNA)are molecules of genetic information, synthesized from a gene readingprocess, that serve as templates for protein synthesis. Proteinexpression is the production of a given protein by a cell. If the genecoding for the given protein, determined from a protein expressionprocess, is enhanced or excess copies of the gene or mRNA are present,the protein may be over-expressed. Conversely, if the gene coding issuppressed or absent, the protein may be under-expressed or absent.

Normal cellular behaviors are precisely controlled by molecularmechanisms involving a large number of proteins, mRNAs, and genes. Geneamplification, gene deletion, and gene mutation are known to have aprominent role in abnormal cellular behaviors through abnormal proteinexpression. The range of cellular behaviors of concern includesbehaviors as diverse as, for example, proliferation or differentiationregulation. Therefore, effective detection and quantification in geneamplification, deletion and mutation, mRNA quantification, or proteinexpression analyses is necessary in order to facilitate useful research,diagnostic and prognostic tools.

There are numerous laboratory techniques directed to detection andquantification in gene amplification, deletion and mutation, mRNAquantification, or protein expression analyses. For example, suchtechniques include Western, Northern and Southern blots, polymerasechain reaction (“PCR”), enzyme-linked immunoseparation assay (“ELISA”),and comparative genomic hybridization (“CGH”) techniques. However,microscopy is routinely utilized because it is an informative technique,allowing rapid investigations at the cellular and sub-cellular levelswhile capable of being expeditiously implemented at a relatively lowcost.

When microscopy is the chosen laboratory technique, the biologicalsamples must first undergo specific detection and revelationpreparations. Once the samples are prepared, a human expert typicallyanalyzes the samples with a microscope alone in a qualitative study, orwith a microscope coupled to a camera and a computer in a quantitativeand generally standardized study. In some instances, the microscope maybe configured for fully automatic analysis, wherein the microscope isautomated with a motorized stage and focus, motorized objectivechangers, automatic light intensity controls and the like.

The preparation of the samples for detection may involve different typesof preparation techniques that are suited to microscopic imaginganalysis, such as, for example, hybridization-based andimmunolabeling-based preparation techniques. Such detection techniquesmay be coupled with appropriate revelation techniques, such as, forexample, fluorescence-based and visible color reaction-based techniques.

In Situ Hybridization (“ISH”) and Fluorescent In Situ Hybridization(“FISH”) are detection and revelation techniques used, for example, fordetection and quantification in genetic information amplification andmutation analyses. Both ISH and FISH can be applied to histological orcytological samples. These techniques use specific complementary probesfor recognizing corresponding precise sequences. Depending on thetechnique used, the specific probe may include a colorimetric (cISH)marker or a fluorescent (FISH) marker, wherein the samples are thenanalyzed using a transmission microscope or a fluorescence microscope,respectively. The use of a colorimetric marker or a fluorescent markerdepends on the goal of the user, each type of marker havingcorresponding advantages over the other in particular instances.

In protein expression analyses, immunohistochemistry (“IHC”) andimmunocytochemistry (“ICC”) techniques, for example, may be used. IHC isthe application of immunochemistry to tissue sections, whereas ICC isthe application of immunochemistry to cultured cells or tissue imprintsafter they have undergone specific cytological preparations such as, forexample, liquid-based preparations. Immunochemistry is a family oftechniques based on the use of a specific antibody, wherein antibodiesare used to specifically target molecules inside or on the surface ofcells. The antibody typically contains a marker that will undergo abiochemical reaction, and thereby experience a change color, uponencountering the targeted molecules. In some instances, signalamplification may be integrated into the particular protocol, wherein asecondary antibody, that includes the marker stain, follows theapplication of a primary specific antibody.

In both hybridization and immunolabeling studies, chromagens ofdifferent colors are used to distinguish among the different markers.However, the maximum number of markers that may be used in a study isrestricted by several factors. For example, the spectral overlapping ofthe colors used to reveal the respective markers may be a limitingfactor because dyes may absorb throughout a large portion of the visiblespectrum. Accordingly, the higher the number of dyes involved in astudy, the higher the risk of spectral overlapping. Further, thespectral resolution of the acquisition device may be a limiting factorand the minimal color shift that the device is able to detect must beconsidered.

In addition, immunochemistry, as well as chemistry in ISH, are generallyconsidered to exhibit poor sensitivity when quantification of a markermust be achieved. However, the quantification accuracy of thesetechniques may be dependent upon several factors. For instance, the typeof reaction used may play a role in the accuracy of the technique sincethe linearity of the relationship between ligand concentration and thedegree of the immunochemical staining reaction may strongly depend onthe reaction type. More particularly, for example, aperoxidase/anti-peroxidase method may be more linear than abiotin-avidin method. The cellular localization of the markers may alsoaffect accuracy where, for example, if membrane and nuclear markersspatially overlap, the resulting color is a mixture of the respectivecolors. Accordingly, since the corresponding quantification issubjective, the accuracy of the determination may be affected. Inaddition, a calibration standard such as, for example, cells with knownfeatures, gels with given concentrations of the marker, or the like, maybe required where a developed analysis model is applied to a new anddifferent case. Staining kits are generally available which incorporatecalibration standards. However, the calibration standard is usually onlyapplicable to a particular specimen, such as a specific cell or astructure of a specific type which is known to exhibit constant featureswith respect to the standard, and may be of limited utility when appliedto a sample of a different nature.

Overall, the described “colorimetric” studies present sample analysisinformation in color and facilitate processing and quantification of theinformation to thereby help to provide a diagnosis or to form aprognosis of the particular case. For illustration, the detection andquantification of the HER2 protein expression and/or gene amplificationmay be assessed by different approaches used in quantitative microscopy.HER2 is a membrane protein that has been shown to have a diagnostic andprognostic significance in metastatic breast cancer. Because HER2positive patients were shown to be more sensitive to treatmentsincluding Herceptin® (a target treatment developed by Genentech), thedefinition of the HER2 status of metastatic breast cancers has beenproven to be of first importance in the choice of the appropriatetreatment protocol. This definition of the HER2 status was based on astudy of samples treated with either hybridization (FISH, ISH) orimmunolabeling (IHC) techniques.

In such studies, using FISH with, for example, an FDA approved kit suchas PathVysion® produced by Vysis, requires an image analysis protocolfor counting the number of copies of the HER2 gene present in everycell. In a normal case, two copies of the gene are found in each cell,whereas more than three copies of the gene in a cell indicate that thegene is amplified. Alternatively, using IHC with, for example, an FDAapproved kit such as Herceptest® produced by Dako, requires an imageanalysis protocol that classified the cases into four categoriesdepending on the intensity and localization of the HER2 specificmembrane staining. Current studies tend to show that these twoinvestigation techniques (hybridization and immunolabeling) may becomplementary and may help pathologists in tumor sub-type diagnosis whencombined.

However, such colorimetry studies require extensive sample preparationand procedure control. Thus, when disposing of adapted stainingprotocols, it is critical to be able to verify that the staining foreach sample matches the particular model used in the image acquisitionand processing device such that useful and accurate results are obtainedfrom the gathered information. Otherwise, the analysis may have to berepeated, starting again from the sample preparation stage, therebypossibly resulting in a costly and time-consuming process.

In a typical microscopy device based on image acquisition andprocessing, the magnified image of the sample must first be captured anddigitized with a camera, prior to analysis. Further, in order to exploitthe color properties of the sample, multispectral image acquisitiondevices and associated multispectral imaging methods have beendeveloped, where such multispectral imaging is typically directed toacquiring multiple images of a scene at different spectral bands. Moreparticularly, charge coupled device (CCD) digital cameras are typicallyused in either light or fluorescence quantitative microscopy.Accordingly, excluding spectrophotometers, two different techniques aregenerally used to perform such colorimetric microscopy studies. In onetechnique, a black and white (BW) CCD camera may be used. In such aninstance, a gray level image of the sample is obtained, corresponding toa monochromatic light having a wavelength specific to the staining ofthe sample to be analyzed. The specific wavelength of light is obtainedeither by filtering a white source light via a specific narrow bandwidthfilter, or by directly controlling the wavelength of the light source,using either manual or electronic controls. Accordingly, using thistechnique, the analysis time increases as the number of colors increasesbecause a light source or a filter must be selected for every differentsample staining or every different wavelength. Therefore, many differentimages of the sample, showing the spectral response of the sample atdifferent wavelengths, must be individually captured in a sequentialorder to facilitate the analysis. When multiple scenes or fields of viewmust be analyzed, the typical protocol is to automate the sequence in abatch mode to conserve processing time.

According to a second technique, a color CCD digital camera is used,wherein three gray level images of the sample are simultaneouslycaptured and obtained. Each gray level image corresponds to therespective Red, Green and Blue channel (RGB) of the color CCD camera.The images are then analyzed directly in the RGB color space byrestricting the analysis to pixels located in a specific region of theRGB cube, the specific region also including pixels from a correspondingtraining database. Alternatively, the images are analyzed, aftermathematical transform of the RGB color space, in one of the many colorspaces defined by the CIE (International Commission on Illumination)such as, for example, an HLS (Hue, Luminance or Saturation) space.Alternatively, some camera manufacturers produce specific CCD cameras,wherein narrow bandwidth filters for targeting specific wavelengths mayreplace the usual Red, Green and Blue filters. In such an instance, thecamera allows a fast image capture of the three spectral components of ascene in a parallel manner. However, cameras modified in this manner maybe restricted to specific spectral analysis parameters because thefilters cannot be changed and therefore cannot be adapted to address aunique dye combination used for the sample. Thus, the second techniquegenerally relies upon either the detection of contrast between thespecies of interest and the remainder of the sample or the analysis ofthe sample over a narrow bandwidth.

Accordingly, techniques used in colorimetric analyses of preparedsamples are of limited use in the detection and quantification ofspecies of interest due to several factors such as, for example,spectral overlapping, mixing of colors due to spatial overlap ofmembrane, cytoplasmic, and nuclear markers, chromatic aberrations in theoptical path, limited spectral resolution of the acquisition device,calibration particularities, subjectivity of the detection andquantification process, and inconsistencies between human operators. Theimage processing portion of colorimetric analysis techniques hashistorically been directed to the subjective detection of contrastwithin the prepared sample or to a complex and voluminous analysis ofthe sample at various specific wavelengths of light using a combinationof light sources and filters.

Thus, there exists a need for a simpler and more effective colorimetricanalysis technique that overcomes detection and quantificationlimitations found in prior art analysis techniques. Such a techniqueshould also be capable of providing high quality data, comprising thenecessary analysis information about the sample, while reducingsubjectivity and inconsistency in the sample analysis.

SUMMARY OF THE INVENTION

The above and other needs are met by the present invention which, in oneembodiment, provides a method of modeling a dye. First, a transmittanceof a sample treated with the dye is determined from a color image of thetreated sample. The image comprises a plurality of pixels and thetransmittance is determined in each of a red, green, and blue channel ofan RGB color space and for each pixel of the image so as to form an RGBtriplet for each pixel. The RGB triplets are thereafter groupedaccording to the minimum transmittance in the red, green, and bluechannels for the respective RGB triplet. Each group of RGB triplets isthen normalized by summing the transmittances in each of the respectivered, green, and blue channels and then dividing each of the summedtransmittances by the number of RGB triplets in the respective group soas to form respective normalized RGB triplets. The normalized RGBtriplets are then tabulated according to the minimum transmittance ofeach normalized RGB triplet so as to form a correspondence table for thedye, wherein the correspondence table extends in transmittanceincrements between 0% and 100% transmittance.

Another advantageous aspect of the present invention comprises a methodof modeling a combination of a plurality of dyes in a video-microscopysystem. First, a correspondence table is formed for each of theplurality of dyes. More particularly, a transmittance of a sampletreated with the respective dye is determined from a color image of thetreated sample. Each image comprises a plurality of pixels and thetransmittance of the respective sample is determined in each of a red,green, and blue channel of an RGB color space and for each pixel of theimage so as to form an RGB triplet for each pixel. The RGB triplets arethen grouped according to the minimum transmittance in the red, green,and blue channels for the respective RGB triplet. Each group of RGBtriplets is normalized by summing the transmittances in each of therespective red, green, and blue channels and then dividing each of thesummed transmittances by the number of RGB triplets in the respectivegroup so as to form respective normalized RGB triplets. Thereafter, thenormalized RGB triplets are tabulated according to the minimumtransmittance of each normalized RGB triplet so as to form thecorrespondence table for the respective dye, the correspondence tableextending in transmittance increments between 0% and 100% transmittance.Once each correspondence table is determined, the correspondence tablesof the plurality of dyes are orthogonally added so as to form a dyespace representation of the plurality of dyes, wherein the dye spacerepresentation has one dimension for each dye and provides a referencemodel for a combination of the plurality of dyes.

Still another advantageous aspect of the present invention comprises amethod of determining an amount of at least one molecular speciescomprising a sample from an image of the sample captured by a colorimage acquisition device, wherein each molecular species is indicated bya dye. First, a dye space representation of a plurality of dyes isformed, each dye having a corresponding correspondence table comprisinga plurality of normalized RGB triplets. In order to form acorrespondence table for a dye, a transmittance of a sample treated withthe respective dye is determined from a color image of the treatedsample. Each image comprises a plurality of pixels and the transmittanceof the respective sample is determined in each of a red, green, and bluechannel of an RGB color space and for each pixel of the image so as toform an RGB triplet for each pixel. The RGB triplets are then groupedaccording to the minimum transmittance in the red, green, and bluechannels for the respective RGB triplet. Each group of RGB triplets isnormalized by summing the transmittances in each of the respective red,green, and blue channels and then dividing each of the summedtransmittances by the number of RGB triplets in the respective group soas to form respective normalized RGB triplets. Thereafter, thenormalized RGB triplets are tabulated according to the minimumtransmittance of each normalized RGB triplet so as to form thecorrespondence table for the dye, the correspondence table extending intransmittance increments between 0% and 100% transmittance. Once eachcorrespondence table is determined, the correspondence tables of theplurality of dyes are then orthogonally added so as to form a dye spacerepresentation of the plurality of dyes, wherein the dye spacerepresentation has one dimension for each dye and provides a referencemodel for a combination of the plurality of dyes.

After the dye space representation is formed, each pixel of the image ofthe sample is compared to the reference model for the combination of theplurality of dyes, wherein the sample is treated by the combination ofthe plurality of dyes and each pixel has a color defined by an RGBtriplet. From the comparison, an optimal combination of normalized RGBtriplets is determined from the respective correspondence tables of thedyes producing the color of the respective pixel, wherein the normalizedRGB triplets of the optimal combination are identifiable according tothe respective dye. An artificial image of the sample, the artificialimage corresponding to the sample image, may then be formed from thenormalized RGB triplets for each dye determined from the optimalcombination. The artificial image thereby indicates a distribution ofthe respective dye over the sample image and facilitates determinationof the amount of the corresponding molecular species.

From the methods disclosed herein, it will be appreciated by one skilledin the art that other advantageous aspects of the present invention maycomprise video-microscopy systems and associated computer softwareprogram products for implementing and accomplishing the describedcapabilities of such methods. For example, a video-microscopy system maycomprises a color image acquisition device capable of capturing amagnified digital image of the sample and a computer device operablyengaged with the image acquisition device for processing the capturedimage. Once the image is captured by the image acquisition device, theimage may be balanced and normalized according to an empty field (white)reference and a black field image and, in some instances, corrected forshading. The image may also be corrected for chromatic aberrations on achannel by channel basis before the individual dyes are analyzed,various protocols are implemented, and histological samples stainedaccording to those protocols are evaluated by the system according tothe methods described herein. Accordingly, the computer device of such asystem may comprise one or more processing portions configured toaccomplish the appropriate analysis of captured images, the processingand storing of the data extracted therefrom, and the subsequent imageprocessing and re-creation of various images utilized by the system.Further, an associated computer software program product is configuredto be executable on such a computer device and may comprise one or moreexecutable portions capable of accomplishing the methodology describedherein, as will be further appreciated by one skilled in the art.

However, though embodiments of the present invention are describedherein, for the sake of example, in terms of a video-microscopy system,it will be understood and appreciated by one skilled in the art that theconcepts describe herein may have a broad applicability tonon-microscopy systems. For instance, the concepts described herein maybe applicable where one or more color components may be separatelycharacterized such that useful information may be gleaned from thedistribution of such color components with respect to an unknown samplecomprising those color components. Further, though the methods, systems,and computer software program products of embodiments of the presentinvention are described herein in conjunction with an image acquisitiondevice, it will be appreciated by one skilled in the art that suchdescription is provided only for a convenient example of one embodimentof the present invention. For example, similar results may be obtainedwith embodiments wherein a system is configured to accept digital sampleimages previously captured or captured by an image acquisition device ofa separate system.

Note also that, when such multi-spectral imaging techniques, asdescribed herein, are particularly adapted to color imaging,substantially real time or video rate processing and viewing of thesample is facilitated. The use of, for example, a RGB color CCD cameraallows acquisition and processing time for sample images to be performedat a video rate, typically 40 millisecond per frame, which provides aconsiderable advantage as compared to prior art imaging techniques whichgenerally exhibit field of view acquisition and processing times of over1 second. Where an RGB camera is used by the system, image acquisitionthrough the different channels is performed in parallel and look-uptables (LUT) can be generated so as to map the spectral characteristicsof each of various dyes. Thus, such capabilities may, for example,enhance processing speed and facilitate real time processing for displaypurposes. Accordingly, an a posteriori evaluation of the image can beperformed to evaluate the efficiency of an a priori known dyecombination model for each pixel. That is, an evaluation as detailedherein also provides a confidence evaluation for each pixel in that thecolor and intensity measured at the given pixel may be justified by acombination of the a priori known dyes.

Thus, embodiments of the present invention comprise a colorimetricanalysis technique for prepared samples that provides effectivedetection and quantification of species of interest that overcomeslimiting factors of prior art techniques such as, for example, spectraloverlapping, mixing of colors due to spatial overlap of membrane andnuclear markers, limited spectral resolution of the acquisition device,calibration particularities, the subjectivity of the detection andquantification process, and inconsistencies between human operators ofthe analysis equipment. Embodiments of the present invention furtherprovide an image processing technique which does not rely upon thesubjective detection of contrast within the prepared sample or a complexand voluminous analysis of the sample at specific wavelengths of lightusing a combination of light sources and filters. Therefore, embodimentsof the present invention provide a simpler and more effectivecolorimetric analysis technique that overcomes detection andquantification limitations in prior art analysis techniques, reducessubjectivity and inconsistency in the sample analysis, and is capable ofproviding the necessary analysis information about the sample, once animage of the sample is captured, without relying upon furtherexamination of the sample to complete the analysis. These and otheradvantages are realized over prior art colorimetric analysis techniquesas described herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

Having thus described the invention in general terms, reference will nowbe made to the accompanying drawings, which are not necessarily drawn toscale, and wherein:

FIG. 1 is a general schematic representation of a quantitativevideo-microscopy system according to one embodiment of the presentinvention.

FIGS. 2A–2C schematically illustrate the addition of RGB triplets to acorrespondence table for a dye according to one embodiment of thepresent invention.

FIG. 3 schematically illustrates the normalization of a correspondencetable according to one embodiment of the present invention.

FIG. 4A illustrates an exemplary model of an NFR dye in a 3D RGB colorspace according to one embodiment of the present invention.

FIG. 4B illustrates an exemplary model of a BCIP-NBT dye in a 3D RGBcolor space according to one embodiment of the present invention.

FIG. 4C illustrates an exemplary model of an NFR dye as shown in FIG. 4Acombined with a BCIP-NBT dye as shown in FIG. 4B in a 3D RGB color spaceaccording to one embodiment of the present invention.

FIG. 5A illustrates an exemplary model of an NFR dye on a 1D scale(correspondence table) according to one embodiment of the presentinvention.

FIG. 5B illustrates an exemplary model of a BCIP-NBT dye on a 1D scale(correspondence table) according to one embodiment of the presentinvention.

FIG. 5C illustrates an exemplary model of an NFR dye as shown in FIG. 5Acombined by orthogonal addition with a BCIP-NBT dye as shown in FIG. 5Bon a 2D scale (dye space) according to one embodiment of the presentinvention.

FIG. 6 is a schematic representation of a dye combination model plottedin a 3D RGB color space showing a pixel from an image of an unknownsample, stained according to the same protocol and plotted against themodel, separated from the model by an error, according to one embodimentof the present invention.

FIG. 7 illustrates an example of a dye combination model plotted in a 3DRGB color space having an overlaid plot of an image of an unknown samplestained according to the same protocol, according to one embodiment ofthe present invention.

FIG. 8 illustrates an example of using a 2D scale (dye space)representation of a combination of two dyes to determine the colorproperties of a pixel of an image of an unknown sample stained accordingto the same protocol, according to one embodiment of the presentinvention.

FIG. 9 is a schematic illustration of a 2D scale (dye space)representation of a combination of two dyes having a defined boundedregion in which a search is conducted for an optimal combination for apixel of an image of an unknown sample stained according to the sameprotocol, according to one embodiment of the present invention.

FIG. 10 is a schematic illustration of a 3D RGB color spacerepresentation of a combination of two dyes having a defined boundedregion in which a search is conducted for an optimal combination for apixel of an image of an unknown sample stained according to the sameprotocol, according to one embodiment of the present invention.

FIG. 11A is an image of an unknown sample stained with a combination ofNFR and BCIP-NBT according to one embodiment of the present invention.

FIGS. 11B and 11C are images of the unknown sample of FIG. 11A showingonly the NFR dye component and only the BCIP-NBT dye component,respectively, according to one embodiment of the present invention.

FIGS. 11D and 11E are images of the unknown sample of FIG. 11A showingonly the gray level NFR dye component and only the gray level BCIP-NBTdye component, respectively, according to one embodiment of the presentinvention.

FIG. 12 is a schematic representation of the practical realization in anextended configuration of a quantitative video-microscopy systemaccording to one embodiment of the present invention.

FIG. 13A is an image of an unknown sample stained with a combination ofNFR and BCIP-NBT according to one embodiment of the present inventionand illustrating an unknown component.

FIG. 13B is an image of a per pixel error distribution of the errorbetween the image as shown in FIG. 13A and the two dye model (NFR andBCIP-NBT) according to one embodiment of the present invention.

FIG. 13C is an image of a per pixel error distribution of the errorbetween the image as shown in FIG. 13A and a three dye model (NFR,BCIP-NBT, and melanin) according to one embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention now will be described more fully hereinafter withreference to the accompanying drawings, in which preferred embodimentsof the invention are shown. This invention may, however, be embodied inmany different forms and should not be construed as limited to theembodiments set forth herein; rather, these embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey the scope of the invention to those skilled in the art. Likenumbers refer to like elements throughout.

Probably the most rapidly increasing cancer appears to be cutaneousmelanoma, for which a new marker has recently been discovered. However,diagnostic and prognostic use requires the precise quantification of thedownregulation of this marker in tumor melanocytes. As such,multispectral analysis is one of the most reliable colorimetricmethodologies for addressing this quantification in routine bright fieldmicroscopy. Accordingly, a technique dedicated to the optimization ofmultispectral analysis and application thereof to the quantification ofthe melanoma marker is described herein, along with associated systemsand computer software programs. More particularly, the present inventionis described herein as being applied to the quantification of themelastatin marker, a promising cutaneous melanoma tumor marker, thoughit will be understood that the particular application is presented forexemplary purposes only and is not to be construed as a restriction orlimitation on the applicability of the present invention. Accordingly,the present invention may be more generally applicable to quantitativecolorimetric studies of a wide range of samples stained by one or moredyes and analyzed according to the methods described herein.

According to the American Cancer Society, the incidence of cutaneousmelanoma is increasing more rapidly than any other cancer in the UnitedStates. For example, there will be about 51,400 new cases of melanoma in2001, 29,000 men and 22,400 women, which is about a 9% increase from2000. In 2001, at current rates, about 1 in 71 Americans will have alifetime risk of developing melanoma. Generally, surgical excision oflocalized primary cutaneous melanoma (American Joint Committee on Cancer[AJCC] stages I and II, comprising approximately 75% of diagnoses) maylead to cure in many patients. Further, the overall 5-year survival ratefor the patients having surgical excision is approximately 80%, therebysuggesting that approximately 20% of stage I and II patients may havemicrometastatic disease at the time of cutaneous melanoma diagnosis.Melanoma is one of the most lethal of cancers. For example, in 1996,melanoma accounted for approximately 2% of all cancer-related deaths inthe United States. Currently, there is no cure for patients havingmelanoma that has metastasized to distant sites, and the median survivalof such patients is only approximately 6 months. Therefore, according toDuncan et al., J. Clin. Oncol., Vol. 19, No. 2, pp. 568–576, 2001, thecontents of which are incorporated herein by reference, identifyingpatients at high risk of developing metastases is one of the mostcritical issues in the management of this form of cancer.

A novel gene, named melastatin, whose expression correlates with in vivoaggressiveness, has been discovered in murine B16 melanoma cell sublineswith divergent metastatic potential in vivo using differential mRNAanalysis. The human melastatin has been cloned and is generallydesignated as MLSN-1. Northern blot analysis has demonstrated melastatinexpression only in melanocytic cells and the choroid (eye tissue), whileno expression of melastatin in other tissues was detected. Though insitu hybridization analysis of human cutaneous melanocytic tissuesrevealed that melastatin mRNA was uniformly expressed in benignmelanocytic nevi, primary cutaneous melanomas showed variable expressionof melastatin mRNA. Notably, melastatin status was correlated with thethickness of the primary tumor, according to Deeds et al., Hum. Pathol.,Vol. 31, No. 11, pp. 1346–1356, 2000, the contents of which areincorporated herein by reference. Preliminary data has also suggestedthat melastatin expression is inversely associated with human metastaticdisease. However, most recent results indicate that downregulation ofmelastatin mRNA in the primary cutaneous tumor is a prognostic markerfor metastasis in patients with localized malignant melanoma and isindependent of tumor thickness and other variables. When used incombination, melastatin status and tumor thickness allow for theidentification of subgroups of patients at high and low risk ofdeveloping metastatic disease.

U.S. Pat. No. 6,025,137 is related to the Melastatin™ gene andmetastatic melanoma and, more particularly, is directed to methods ofdetecting Melastatin™ in patient tissue samples in order to determinewhether a patient has, or is at risk for developing, metastaticmelanoma. In order to use Melastatin™ expression as a diagnostic markerin routine work, evaluation must be conducted with reliability and atlow cost. IHC and ISH on histological material from skin samples aretypically the most convenient techniques that are routinely used toquantify protein expression at low cost. Thus, embodiments of thepresent invention are directed to the quantification of melastatinstaining in both normal melanocyte nuclei (melanocytes from the basallayer of epithelial cells), considered as reference nuclei, and abnormalmelanocyte nuclei (melanocytes from tumor foci). The results of such aquantitative analysis indicate whether the gene is either downregulatedor normally expressed in the abnormal nuclei. However, the efficiency ofthe quantitative analysis heavily depends upon the image analysismethodology, which must consider and perform segmentation of themelanocyte nuclei, as well as colorimetric analysis of the specific dyesused in the protocol.

The platform for the evaluation of biological samples via image analysisis increasingly shifting from a general-purpose image analyzer to amore, and often highly, specialized dedicated “pathology workstation.”Such workstations are typically designed to facilitate routine work,often combining many of the tools needed to provide a pathologist withthe necessary information to determine the best possible results. Oneexample of such a workstation is illustrated in FIG. 1 as a quantitativevideo-microscopy system, indicated by the numeral 100, according to oneembodiment of the present invention. The system 100 generally comprisesa microscope 150 having a light source 200 and a magnifying objective250, a camera 300, a computer device 350, and a data transmission link400 between the camera 300 and the computer device 350. The microscope150 may comprise, for example, an Axioplan (or Axiovert) microscopeproduced by ZEISS of Germany or a similar microscope having a brightfield light source. The camera 300 operably engages the microscope 150and, in one embodiment, comprises a 3CCD RGB camera such as, forinstance, a Model No. DC-330E Dage-MTI RGB 3CCD camera produced byDage-MTI, Inc. of Michigan City, Ind. or a similar RGB camera.Typically, such a camera 300 also includes an associated frame grabber(not shown) to facilitate image capture, both the camera 300 andassociated frame grabber being referred to herein as the “camera 300”for convenience. In some instances, both camera 300 and microscope 150may be replaced by, for example, a linear flat scanner having a 3CCDchip or equivalent and a controlled illumination source. For instance, aModel No. Super CoolScan 4000 ED scanner produced by Nikon Corporationmay be used for low-resolution imaging. Note that, though differentconfigurations of the necessary system 100 are contemplated by thepresent invention, the present invention will be described herein interms of a camera 300 and associated microscope 150. Accordingly, oneskilled in the art will understand and appreciate the capabilities andmethodologies associated with these different configurations foraccomplishing the present invention as detailed herein.

The camera 300 is generally configured to capture an image 450 of asample 500 through the magnifying objective 250 (where a flat scanner isused, the image 450 is captured through internal lenses), wherein theimage 450 may further comprise a digital image having correspondingimage data (collectively referred to herein as “the image 450”). Theimage 450 is generally captured as a whole, wherein the correspondingimage data comprises a red channel 550, a green channel 600, and a bluechannel 650 image of the field of view. The data transmission link 400is configured so as to be capable of transmitting the image 450 to thecomputer device 350, wherein the computer device 350 is furtherconfigured to be capable of analyzing the image 450 with respect to eachof the red 550, green 600, and blue 650 channels.

The quantitative video-microscopy system 100 of the present invention,when evaluating a melastatin expression, must meet certain requirementsin order to provide usable results, as will be described in furtherdetail herein. For example, staining of thick histological samples mayinduce non-linearity in the relationship between the intensity of thestaining and the protein concentration in the sample. In such aninstance, the integrated optical density of the sample will not providestable, reproducible colorimetric data and thus an alternative measuremust be used. In addition, 3 chromagens are generally present in thehistological samples: the marker (BCIP-NBT), a morphologicalcounterstain (Nuclear Fast Red—NFR), and natural melanin. All threechromagens must be taken into account for several reasons. First, inorder to realize a quantitative analysis of the tumor marker when usinga counterstain for morphological analysis and melanocyte recognition, aspecific spectral analysis is required in order to distinguish andquantify both the marker and the counterstain. Further, where thecounterstain is applied to the sample after the marker, cells highlystained with the marker are less easily stained by the counterstain.Also, melastatin is used as a downregulation marker and, because of highbackground noise, it is often problematic to discriminate non-specificbackground noise from downregulated marker specific staining. Inaddition, melanin (brown) may also be a potential source of interferenceduring evaluation of the marker in regions of particularly intensestaining.

Thus, according to one advantageous aspect of the present invention, aspectral analysis of the melastatin expression is based on computationof a model of each dye used in the protocol. As previously described, ina melastatin expression, three distinct spectral components aretypically involved: the counterstain (NFR); the melastatin marker(BCIP-NBT); and a natural pigment (melanin) of the histological sample.Accordingly, the spectral properties of each of the different spectralcomponents are first studied separately with the video-microscopy system100 of the present invention using, for example, one histological slideper dye. That is, in the training phase of the video-microscopy system100, it is presumed that if only one dye is being analyzed by the system100, then the dye will be homogeneously and continuously distributed inan RGB color space. Once the histological sample 500 for a particulardye is prepared, an image 450 of the sample 500 is captured with thecamera 300 through the magnifying objective 250. The resulting digitalimage 450 is an ordered two-dimensional (2D) set of pixels, wherein eachpixel is defined by an RGB triplet. That is, if an 8 bit color imageacquisition device such as, for example, an 8 bit 3CCD RGB camera 300 isutilized, each pixel may be defined by a measured light intensity I ortransmittance of the sample 500 in each of the red 550, green 600, andblue 650 channels, the measured light having 256 possible values rangingbetween limits of 0 and 255.

When at least one image 450 of a histological sample 500 for each dyehas been captured, then artificial images for each dye in a separateunknown sample, stained according to the same protocol, may bereconstructed, as will be discussed in further detail below. Typically,the image 450 of the histological sample 500 for each dye is analyzed bythe computer device 350 on a per pixel basis so as to provide acorresponding data set of RGB triplets. An RGB color space may then beestablished, the RGB color space comprising a cube having an axis foreach of the red 550, green 600, and blue 650 channels, with each axisextending from 0 to 255 for an 8 bit image acquisition device 300.Accordingly, it has been found that, when the RGB triplets are plottedin the RGB color space to illustrate the spatial distribution of the dyewithin that space, the dye occupies a characteristic path leading fromblack (0% transmittance or no intensity in each of the red 550, green600, and blue 650 channels) to white (100% transmittance or fullintensity in each of the red 550, green 600, and blue 650 channels) inthe RGB color space. Note that the RGB properties corresponding to 0%transmittance and 100% transmittance may readily determined for aparticular system and, accordingly, typically comprise known values. Ithas also been found that the same analysis performed for furtherhistological samples 500 stained with the same dye reveals areproducible spectral pattern, more particularly the characteristic path940, for that dye in the RGB color space (see, for example, FIG. 4A forNFR and FIG. 4B for BCIP-NBT). Thus, a spectral pattern for therespective dye may be stored in a correspondence table, also referred toherein as a look-up table (LUT), describing the RGB properties of a dyefrom 0 to 100% transmittance in the RGB color space.

A typical correspondence table for a dye in an 8 bit image acquisitionsystem may be defined with, for example, 256 rows and 4 columns, whereineach row 900 represents an intensity or transmittance increment. Threeof the columns 905, 910, 915 represent each of the red 550, green 600,and blue 650 channels, while the fourth column 920 provides forindexing, as described further below. Each RGB triplet for thecorresponding pixel in the image 450 is then entered in thecorrespondence table and indexed according to the minimum intensity ortransmittance value of the triplet, the minimum intensity correspondingto the most absorbing (least intensity or transmittance) channel for theparticular dye. Further, for each RGB triplet added to thecorrespondence table in the appropriate indexed row 900, the intensityvalues of the RGB triplet in the red 550, green 600, and blue 650channels are added to the respective intensity values for any RGBtriplet already inserted into the indexed row 900. In addition, forevery RGB triplet added to an indexed row 900, the index in the fourthcolumn 920 of the correspondence table is incremented by one so as totrack the number of pixels defined by the same indexed row 900 in thecorrespondence table. This process is shown in, for example, FIGS. 2A–2Cand may be, for instance, applied to a large set of images for each dyeso as to provide an extensive and accurate spectral characterization ofthat dye. Thus, once all of the pixels in the set of images for that dyehave been evaluated and inserted into the correspondence table accordingto the corresponding RGB triplet for respective pixel, thecorrespondence table is finalized. Thereafter, as shown in FIG. 3, thefinalized correspondence table is normalized, wherein, for each indexedrow 900, the summed intensity value in each of the red 550, green 600,and blue 650 channels is divided by the number of RGB triplets collectedin that row, as indicated by the index value in the fourth column 920 ofthe correspondence table. Thus, the normalized RGB triplets 925, 930,935 in the correspondence table comprise a representative line 945through the RGB color space for the particular dye (see, for example,FIG. 4A for NFR and FIG. 4B for BCIP-NBT), wherein the representativeline 945 extends along the previously determined characteristic path 940for that dye.

Once normalized, the indexed rows will generally include normalized RGBtriplets for the respective row. However, in some instances, some of theindexed rows may be missing an RGB triplet since, for example, anindexed row in the correspondence table may not necessarily berepresented by a corresponding pixel in the image. That is, there may beinstances where no RGB triplet for a pixel in the image meets thecriteria for being added to a particular row. Further, in someinstances, a significance criteria may be established so as to eliminateartifacts due to under-representation in the image. For example, anindexed row in the correspondence table may only be consideredsignificant if 1000 or more pixels of the image have been indexed inthat row. If an indexed row fails to meet the significance criteria,then any values in the three R, G, and B columns may be discarded orotherwise removed from the model for that dye. Accordingly, in instanceswhere an indexed row of the normalized correspondence table does notinclude a normalized RGB triplet, an interpolation may be used todetermine an approximated RGB triplet for any row lacking a normalizedRGB triplet. Typically, the number of contiguous missing orinsignificant RGB triplets is small and thus, for example, a cubicspline interpolation (see e.g., Numerical Recipes in C: The Art ofScientific Computing (ISBN 0-521-43108-5), pp. 113–117, Copyright (c)1988–1992 by Cambridge University Press. Programs Copyright (c)1988–1992 by Numerical Recipes Software) may be implemented so as toobtain an approximated RGB triplet for an empty row from both a knownnormalized RGB triplet at higher transmittance indexed row and a knownnormalized RGB triplet at a lower transmittance indexed row. However,one skilled in the art that other forms of interpolation may be equallyapplicable and effective depending on the requirements of the particularapplication. For example, a linear interpolation may be used in someinstances. Accordingly, once the correspondence table has beencompleted, the respective dye is sufficiently modeled and the model iscapable of being used in further applications as detailed below.

According to one advantageous embodiment of the present invention, onceindividual dyes have been modeled according to the procedure describedabove, two or more of these models for individual dyes may be combinedso as to correspond to the staining protocol used for a particularexpression. Thus, for example, for a melastatin expression, individualmodels for the counterstain (NFR) and the melastatin marker (BCIP-NBT)may be combined so as to facilitate evaluation of an unknownhistological sample stained according to this protocol. Generally, therespective correspondence table for each dye defines transmittancevalues ranging from black (0% transmittance) to white (100%transmittance) with linear incremental steps (1/256% in transmittancefor an 8 bit/channel image acquisition device) therebetween. Thus, aspreviously described, the LUT for a single dye is the representativeline 945 extending through the characteristic path 940 for that dye inthe RGB color space, wherein the representative line 945 may beexpressed in a one-dimensional (1D) scale summarizing the spatialdistribution of the dye in the RGB color space from 0 to 100%transmittance (See, for example, FIG. 5A for NFR and FIG. 5B forBCIP-NBT).

For a staining protocol specifying a certain combination of dyes, amodel of that protocol for that dye combination may be realized from theorthogonal addition of the LUTs for the respective dyes, as shown, forexample, in FIG. 5C. Accordingly, such a model would be defined ashaving one dimension per dye. Generally, the addition of, for example,two dyes such as those used in a melastatin expression, may beaccomplished using the Lambert-Beer law to generate the appropriate RGBtriplets for the particular dye combination. Thus, according to aparticularly advantageous aspect of the present invention, the system100 is configured to analyze the sample in accordance with theLambert-Beer law. The Lambert-Beer law generally describes aproportionality that can be observed between the concentration ofmolecules in a solution (the concentration of the “molecular species” orthe “sample”) and the light intensity measured through the solution, andis typically expressed as:OD=ε·l·C  (1)where OD is the optical density of the solution, ε is a proportionalityconstant called the molar extinction or absorption coefficient, l is thethickness of the sample, and C is the concentration of the molecularspecies. The absorption coefficient ε is specific to the molecularspecies and is typically expressed in units of 1·mol⁻¹·cm⁻¹.

This proportionality relationship defined by the Lambert-Beer law hasbeen verified under the several conditions including, for example,monochromatic light illuminating the sample, low molecular concentrationwithin the sample, generally no fluorescence or light responseheterogeneity (negligible fluorescence and diffusion) of the sample, andlack of chemical photosensitivity of the sample. Further, anotherrequirement for an analysis according to the Lambert-Beer law includes,for instance, correct Koehler illumination of the sample under themicroscope. Koehler illumination is available with many modemmicroscopes, providing an even illumination of the sample in the imageplane and allowing for effective contrast control. Koehler illuminationis critical for certain processes such as, for example, densitometryanalysis. Correct Koehler illumination is typically provided by, forexample, a two-stage illuminating system for the microscope in which thesource is imaged in the aperture of the sub-stage condenser by anauxiliary condenser. The sub-stage condenser, in turn, forms an image ofthe auxiliary condenser on the object. An iris diaphragm may also beplaced at each condenser, wherein the first iris controls the area ofthe object to be illuminated, and the second iris varies the numericalaperture of the illuminating beam.

The Lambert-Beer law has an additive property such that, if the samplecomprises several light-absorbing molecular species, for example, s₁ ands₂, having respective concentrations C₁ and C₂, the OD of a sample ofthickness l (where l₁=l₂=l for the sample, as indicated in the solutionhereinafter) can be expressed as:OD=ε ₁·l₁ ·C ₁+ε₂ ·l ₂ ·C ₂  (2)This situation may occur, for example, in a biological analysis where a“scene,” a field of view, or a portion of the sample has been stainedwith two dyes consisting of a marker dye for targeting the molecularspecies of interest and a counterstain for staining the remainder of thesample.

Once the microscope 150 has been configured to provide Koehlerillumination for image acquisition and chromatic aberrations have beenaddressed, the additive property of the Lambert-Beer law can be appliedto chromagen separation. For instance, the additive property of theLambert-Beer law can be expanded to a situation in which the scene isanalyzed in a color environment, generated by, for example, an RGBcamera, separated into a red, green, and blue channel. In such aninstance, the marker dye (or “dye 1”) exhibits absorption coefficients,ε_(1r), ε_(1g), and ε_(1b), in the red, green and blue channels,respectively. Note that, in some instances, the analysis of the image ineach of the red, green, and blue channels is equivalent to analyzing ared representation of the image across the red spectra, a greenrepresentation of the image across the green spectra, and a bluerepresentation of the image across the blue spectra. Accordingly, thecounterstain (or “dye 2”) exhibits absorption coefficients, ε_(2r),ε_(2g), and ε_(2b), in the red, green and blue channels, respectively.Therefore, according to the additive property of the Lambert-Beer law,analysis of the sample in the RGB environment leads to three equationsfor the optical density thereof:OD _(r)=ε_(1r) ·l ₁ ·C ₁+ε_(2r) ·l ₂ ·C ₂  (3)OD _(g)=ε_(1g) ·l ₁ ·C ₁+ε_(2g) ·l ₂ ·C ₂  (4)OD _(b)=ε_(1b) ·l ₁ ·C ₁+ε_(2b) ·l ₂ ·C ₂  (5)where OD_(r), OD_(g), and OD_(b) represent the optical densities of thesample measured in the red, green and blue channels, respectively. Stillfurther, in the case of increased sample preparation complexity such as,for example, the treatment of the sample with three different dyes,equations (3), (4), and (5) become:OD _(r)=ε_(1r) ·l ₁ ·C ₁+ε_(2r) ·l ₂ ·C ₂₊ε_(3r) ·l ₃ ·C ₃  (6)OD _(g)=ε_(1g) ·l ₁ ·C ₁+ε_(2g) ·l ₂ ·C ₂₊ε_(3g) ·l ₃ ·C ₃  (7)OD _(b)=ε_(1b) ·l ₁ ·C ₁+ε_(2b) ·l ₂ ·C ₂₊ε_(3b) ·l ₃ ·C ₃  (8)In such a situation, the three dyes may comprise, for instance, onemarker dye and two counterstains, or two marker dyes and onecounterstain, or even three separate marker dyes. For example, incontinuance of the Melastatin™ expression application, the three dyescomprise the counterstain (Nuclear Fast Red), the marker (BCIP/NBT) andthe natural pigment (melanin) of the histological sample. It will beunderstood by one skilled in the art, however, that this demonstratedproperty of the Lambert-Beer law may be expanded to included an evengreater plurality of dye combinations in accordance with the spirit andscope of the present invention. Also note that, in some instances, oneor more of the dyes may be initially excluded from the describedanalysis and then later characterized and added to the model for thecombination of dyes, where necessary. For example, some samples may becharacterized as having few or localized cells which express melanin.Accordingly, for many images of that sample, a model consisting only ofthe counterstain (Nuclear Fast Red) and the marker (BCIP/NBT) may besufficient to analyze those images. However, where cells expressingmelanin are present, the two-dye model may be supplemented with a thirddye component corresponding to melanin, the third dye component beingobtained and incorporated into the model as described herein. Thesufficiency of the particular model and instances requiring theincorporation of additional dye components are further discussed belowwith respect to the concept of determining an error between respectivepixels of the image of the sample and the applicable dye model.

One particularly advantageous aspect of the present invention utilizes afast capture color imaging device such as, for example, a 3CCD RGBcamera, for multi-spectral imaging of the markers over three distinct(red, green, and blue) channels so as to facilitate multi-spectralimaging of biological markers. Accordingly, the application of theLambert-Beer law to a digital microscopy system 100 of the presentinvention recognizes that the Lambert-Beer law can also be expressed as:OD _((x,y))=log I _(0(x,y))−log I _((x,y))  (9)for a digital image 450 of the sample 500 comprising a plurality ofpixels arranged, for example, according to a Cartesian coordinatesystem, where (x,y) signifies a particular pixel in the image 450,OD_((x,y)) is the optical density of the sample 500 at that pixel,I_((x,y)) is the measured light intensity or transmittance of the sample500 at that pixel, and I_(0(x,y)) is the light intensity of the lightsource 200 as measured without any intermediate light-absorbing object,such as the sample. It will further be appreciated by one skilled in theart that the logarithmic relationship described in equation (9) may beexpressed in various bases within the spirit and scope of the presentinvention. For example, the relationship may be expressed in base 2,base 10, or natural logarithms, wherein the various bases are related byrespective proportionality constants (for example, ln(x) orlog_(e)(x)=2.3026 log₁₀(x)). Thus, the proportionality constant may beappropriately considered where relative comparisons are drawn in lightintensities. Further, in quantitative microscopy according to theLambert-Beer law, the proportionality relationship between the opticaldensity OD of the sample and the dye concentrations is conserved.Therefore, for a prepared sample 500 examined by the system 100, theappropriate relation is expressed as:ln I ₀ −ln I=ln I ₀ /I=OD=ε·l·C  (10)

Where, for example, an 8 bit RGB camera 300 is used in the system 100,the light intensity transmitted through the sample in each channel maybe expressed as 2⁸(=256) values between 0 and 255. For example, theinitial intensity I₀ of the light source 200, which corresponds to 100%transmittance, will preferably be expressed in each of the red 550,green 600, and blue 650 channels as a value approaching 255,representing the brightest possible value in each channel. The camera300 and/or the light source 200 may be adjusted accordingly such that,in the absence of the sample, a pure “white” light will have anintensity value of 255 in each of the red 550, green 600, and blue 650channels, corresponding to 100% transmittance. Conversely, in theabsence of light, generally corresponding to transmittance approaching0%, a “black image” will have an intensity value approaching 0 in eachof the red 550, green 600, and blue 650 channels. At any pixel, theinitial intensity I₀ of the light source 200, corresponding to 100%transmittance, is therefore expressed as the difference between theintensity value measured in presence of the light source 200 minus theintensity value measured in absence of the light source 200 for each ofthe red 550, green 600, and blue 650 channels. Because the intensity ofthe light source 200 may vary spatially across the image 450, or overthe measured field of view, and because the magnifying objective 250 orother optical components may heterogeneously absorb light, 100%transmittance may be represented by various differential intensitiesover the measured field of view. However, since the optical density ODof the sample is expressed as the logarithm of the ratio of lighttransmittance in absence of the sample (initial intensity I₀) to lighttransmittance in presence of the sample (I), the optical density OD islargely spatially insensitive to small variations in the differentialintensities over the measured field of view. Since the light source 200remains substantially constant over time, or can be easily re-evaluated,the measurement of the light intensity for any pixel, in the presence ofthe sample, can be translated into the transmittance I at that pixel andin each of the red 550, green 600, and blue 650 channels. Once valuesfor the initial intensity I₀ and transmittance I are determined, theoptical density OD can be computed.

In the case of a protocol including two dyes such as, for example, themelastatin expression, the model for the combination of the two dyes isa 2D map corresponding to the orthogonal addition of the LUT for one dyewith the LUT for the second dye, each LUT being expressed as aone-dimensional (1D) scale (FIGS. 5A and 5B) describing therepresentative line 945 extending through the characteristic path 940for that dye in the RGB color space from 0 to 100% transmittance. Eachpixel at an (x,y) location on the 2D map (FIG. 5C) thus corresponds tothe addition of dye 1 (i.e. NFR) and dye 2 (i.e. BCIP-NBT). That is,each pixel at an (x,y) location on the 2D map corresponds to thecombination of an RGB triplet from the LUT of dye 1 at indexed row xwith an RGB triplet from the LUT of dye 2 at indexed row y. Accordingly,the resulting RGB triplet for the pixel at (x,y) in the 2D map may beexpressed as:R _((x,y))=255/exp(OD _(R(x,y)))  (11)G _((x,y))=255/exp(OD _(G(x,y)))  (12)B _((x,y))=255/exp(OD _(B(x,y)))  (13)where R_((x,y)), G_((x,y)), and B_((x,y)) are transmittances in each ofthe red 550, green 600, and blue 650 channels, respectively. However,though transmittances generally do not exhibit an additive property,optical densities do exhibit an additive property. Accordingly, for an 8bit/channel system, the optical densities in each of the red 550, green600, and blue 650 channels for the pixel at (x,y) in the 2D map may beexpressed as:OD _(R(x,y)) =OD _(Rx) +OD _(Ry) =Ln(255/R _(x))+Ln(255/R _(y))  (14)OD _(G(x,y)) =OD _(Gx) +OD _(Gy) =Ln(255/G _(x))+Ln(255/G _(y))  (15)OD _(B(x,y)) =OD _(Bx) +OD _(By) =Ln(255/B _(x))+Ln(255/B _(y))  (16)As such, because every pixel in the 2D model is defined by an RGBtriplet determined from the combination of an appropriate RGB tripletfor each of the two dyes, the 1D model of each dye, as well as the 2Dmodel of combination of the two dyes, may be separately displayed in a3D RGB color space, as shown in FIG. 4C.

After the model for the combination of dyes used for the particularprotocol has been completed, the model may then be applied to unknownhistological samples stained according to that protocol. Accordingly,the system 100 is then used to capture an image 450 of an unknownhistological sample 500 stained according to a protocol, such as themelastatin expression, for which an appropriate model has beenpreviously developed. As with the dye modeling procedure, the image 450of the unknown histological sample 500 is analyzed on a per pixel basis.Thus, each pixel P_(i) in the image 450 of the unknown histologicalsample 500 is defined by an RGB triplet (R_(i), G_(i), B_(i)) and oneparticularly advantageous aspect of the present invention recognizesthat such a pixel P_(i) can be correlated with a color P_(m), defined byan RGB triplet (R_(m), G_(m), B_(m)), in the model. The color P_(m)corresponds to the dye combination which minimizes the Euclideandistance between the pixel P_(i) and the model in the RGB color space,as shown in FIG. 6. P_(m) is thereby determined by minimizing thesolution of d(P_(i), P_(m)) over the model, wherein:d(P _(i) ,P _(m))=SQRT((R _(i) −R _(m))²+(G _(i) −G _(m))²+(B _(i) −B_(m))²)  (17)Further, the minimized Euclidean distance between P_(i) and P_(m)corresponds to an error between the real image and the model and thusindicates the efficiency of the model in describing the real image. Sucha concept can be graphically displayed in the RGB color space and may beused to evaluate the sample, as shown, for example, in FIG. 7. Moreparticularly, the mean error of the pixels representing a specificobject (i.e. the mask of a nucleus) within the image can be used toeither accept or reject the model for that specific object. According tothis procedure, objects within the image may be identified as includingdyes other than the dyes used to construct the model. For example, inthe melastatin expression example, a significant mean error in a portionof the image of the sample may indicate that the structure being studiedin the image comprises the natural pigment (melanin) which was notconsidered in developing the model of the two other dyes, contamination,or an unidentified artifact. In contrast, a small or nonexistent errormy indicate that the portion of the sample being studied includes tissuewhich is stained by any or all of the dyes used to develop the model forthat protocol. Further, for instance, a significant constant error foundacross all of the pixels in the image of the unknown histological samplemay indicate that the model for the protocol has shifted compared to theunknown histological sample. In still other instances, significant erroracross all of the pixels of the image may indicate that one or moreimproper dyes have been used to stain the unknown histological sample.

In instances where the source of a significant mean error isidentifiable, for example, from an examination of the image, the dyemodel may be supplemented with the addition of a further dye componentso as to provide an improved model for analyzing the image or an objectwithin the image. For instance, some images may include cells expressingmelanin, which may be identified as such by one skilled in the artviewing those images. Such an image is shown, for example, in FIG. 13A,wherein the melanin component is indicated by the brownish component ofthe image. Accordingly, for a dye model comprising the counterstain(Nuclear Fast Red) and the marker (BCIP/NBT), an error distribution,such as shown in FIG. 13B, indicates the melanin component in red,whereas the blue component indicates areas of the image sufficientlydescribed by the two dye model so as to exhibit little or no error forthose pixels. In such instances, a separate melanin source or sample maybe characterized as detailed herein and the results then added toexisting dye model. That is, for example, the Melastatin™ expressiondescribed herein may be analyzed by a three dye model comprising acomponent for the counterstain (Nuclear Fast Red), a component for themarker (BCIP/NBT), and a component for the natural pigment (melanin)where an image of the stained histological sample exhibits cellsexpressing melanin. FIG. 13C illustrates an error distribution analysisof the image shown in FIG. 13A with the melanin component added to thetwo dye model so as to produce a three dye model, the predominantly blueerror image indicating that the three dye model is now sufficient todescribe all of the pixels in the image. Note also that one skilled inthe art will appreciate that adding the additional dye component to thedye model may be readily accomplished as detailed herein.

Once an error has been determined for each pixel of the image, furtheranalysis of these errors may provide other useful information withrespect to the image. For example, for as given object, such as a cell,within the image, the errors for the pixels comprising the object may beintegrated to provide an indication of whether the object is stainedwith the dyes comprising the model so as to evaluate whether that objectis valid for further assessment. In some instances, the evaluated imagemay be configured so as to provide for verification of the particulardye model, as will be appreciated by one skilled in the art. Further, ananalysis of the errors may be structured so as to, for example, providefor detection of objects within an image that are not described by theparticular dye model, indicate variations in the preparation of multiplesamples, and variations within a single sample. Thus, the evaluation oferror between the image and the model may advantageously providesignificant and valuable information for evaluation of the sample asdescribed herein.

Once the pixels of the image of the unknown histological sample havebeen captured and evaluated by the system 100, the pixels may beevaluated according to the respective corresponding color P_(m) todetermine the appropriate RGB triplet contributed by each dye in the 2Dcombination model (i.e. (R_(BCIP), G_(BCIP), B_(BCIP)) and (R_(NFR),G_(NFR), B_(NFR))) from the LUT of the respective dye, as previouslydescribed and as shown, for example, in FIG. 8. As such, the image ofthe unknown histological sample may then be displayed as the capturedimage and as individual images each representing the distribution of arespective dye. As previously described, each dye has a concentrationrepresented by a transmittance ranging from 0% to 100% transmittance,the transmittance range being further represented by gray values of 0 to255 for an 8 bit/channel system. As such, since the 2D combination modelis used to evaluate the image of the unknown histological sample,whenever a pixel color P_(m) (RGB triplet) in the image of the unknownsample is not present in the dye combination model, as manifested in thecalculation of an error greater than zero, the proposed solution must beverified with respect to the theory of optical densities and theLambert-Beer Law. That is, when an RGB value for a pixel in the image ofthe unknown histological sample does not correspond to the dyecombination model, that RGB value may include some noise from, forexample, the histological sample, the camera, the optical systems, orother noise source. In addition, or in the alternative, the errant RGBvalue for the pixel may indicate that the RGB value was generated from adifferent dye combination as compared to the dye combination model.

Accordingly, in order to ensure robustness and adequacy of the dyecombination model, one advantageous aspect of the present inventionutilizes a particular approach, as shown in FIG. 9. First, the dyecombination model space investigated during the search for the optimaldye combination is restricted to a bounded region 950 with respect tothe investigated pixel of the image of the unknown histological sample.More particularly, the bounded region 950 in which to determine theoptimal dye combination for the investigated pixel is defined by the LUTindexed row 955, 960 for each individual dye that separates RGB valueshaving equal or higher transmittance with respect to the investigatedpixel from RGB values having a lower transmittance lower with respect tothe investigated pixel in any of the red 550, green 600, and blue 650channels. That is, the LUT indexed row defining the bounded region 950for the respective dye cannot have a transmittance in any of the red550, green 600, and blue 650 channels that is lower than thetransmittance in the corresponding channel of the investigated pixel.The bounded region 950 is thus defined on the premise that the searchfor the optimal dye combination cannot be conducted among dyeconcentrations, for any dye and in any of the red 550, green 600, andblue 650 channels, that would individually exhibit a higher opticaldensity than the investigated pixel. As such, the bounded region 950 mayalso be indicated on an error map in the RGB color space, as shown inFIG. 10, illustrating, for any pixel, the distance of the investigatedpixel to the dye combination model. Solving for the optimal dyecombination for each pixel of the image of the unknown histologicalsample therefore facilitates, for example, visualization andquantification of the amount of each dye (counterstain, marker ornatural pigment) distributed over the image. For example, the image ofthe sample, as shown in FIG. 11A, may be shown as an artificial image ofonly one of the component dyes (FIG. 11B for NFR only and FIG. 11C forBCIP-NBT only). Such artificial images for each of the component dyesare typically reconstructed on a per pixel basis from the LUT for eachrespective dye. That is, the artificial image for that single dyecomponent is constructed on a per pixel basis from the actualtransmittance value in each of the red, green, and blue channels of theappropriate index row of the LUT for that dye, wherein the appropriateindex row for the respective pixel corresponds to the contribution ofthat dye as determined from the optimal dye combination for that pixel.Alternatively, the original image (FIG. 11A) may be shown as anartificial gray level image of only one of the component dyes (FIG. 11Dfor NFR only and FIG. 11E for BCIP-NBT only). Such gray level artificialimages are also typically reconstructed on a per pixel basis from theLUT for each respective dye. However, in these instances, the gray levelartificial image for the single dye component is constructed on a perpixel basis from the appropriate index row of the LUT for that dye,wherein the appropriate index row for the respective pixel correspondsto the contribution of that dye as determined from the optimal dyecombination for that pixel. More particularly, as previously described,the index row represents the RGB channel with the least transmittancewhich, when combined with the initial intensity I₀, allows theparticular pixel to be expressed in terms of an optical density OD withthe gray level artificial image being constructed accordingly, as willbe appreciated by one skilled in the art. One skilled in the art willalso appreciate that, for example, subsequent quantification of each dyecomponent within the image, or other qualitative or quantitativeanalysis of an image, may be further undertaken once the artificialimages have been determined using various forms of image processing orother techniques commonly used in image analysis and such subsequentanalysis, though not described in detail herein for the sake of brevity,will be considered to be within the scope of the embodiments of theinvention as presented herein.

As previously discussed, though embodiments of the present invention aredescribed herein, for the sake of example, in terms of avideo-microscopy system, it will be understood and appreciated by oneskilled in the art that the concepts describe herein may have a broadapplicability to non-microscopy systems. For instance, the conceptsdescribed herein may be applicable where one or more color componentsmay be separately characterized such that useful information may begleaned from the distribution of such color components with respect toan unknown sample comprising the color components. Such instances may bepresent where, for example, one or more known liquid may becharacterized, as disclosed herein, and those characterizations thenapplied to the determination of the characteristics of an unknown liquidsolution comprised of the individual known liquids, such liquidscomprising, for example, paints or stains. In a more general sense, theconcepts described herein may be applicable where digital imagecharacterization of individual components may be used to analyze animage of an unknown sample, which may not necessarily be comprised ofthe known individual components, wherein much information may beobtained from a comparison of the unknown sample to the selected model,as will be appreciated by one skilled in the art.

Further advantageous aspects of the present invention are realized as aresult of the dye identification techniques using color video imaging aspreviously described herein. For example, an artificial image of thefield of view may be generated in an RGB color space or in gray levelsas a substantially real time or live image, or as a still image, forcombinations of the dyes comprising a marker and/or a counterstain usedfor the staining protocol for the sample. More particularly, anartificial image of the field of view may be produced which shows thesample as affected by all of the dyes, the sample as affected by one ormore marker dyes, or the sample as affected by the counterstain.Consequently, since the dyes used to prepare the sample arecharacterized by the system, the capabilities of the system may beextended such that, for instance, the sample or field of view may beautomatically scanned to detect a specific region of interest asidentified by the characteristics of a particular dye (or lack of thosecharacteristics) or to affect or facilitate a task to be performed onthat specific region of interest.

According to one embodiment of the present invention, the system may beconfigured so as to be capable of detecting one or more particular dyeswhich have been previously characterized by the system. In someinstances, such a dye may comprise, for example, the ink from aparticular pen or similar ink marker that has been characterized by thesystem as having unique color features, these unique color featuresbeing retained by the system as a corresponding LUT for that dye. Itfollows that the system may be configured to recognize and respond toportions of the field of view in which this dye is identified and that,in some instances, the one or more particular markers may comprise atangible portion of such a system as described herein. For instance,such a pen may be used, for example, where an operator such apathologist or a cytotechnologist identifies special areas of intereston a sample-containing glass or plastic slide. A special area ofinterest may comprise, for example, a potential diagnostic area or areference area. The operator, using the pen, may then surround the areawith a line of ink from that pen. After processing a number of slides,the operator may feed the slides into, for instance, an automaticscanning system for quantitative evaluation. Having been configured todetect the ink from the pen, the system may then inclusively identifythe area of interest, corresponding to the area within the ink line,circled by the operator with the pen. The system may thereafterappropriately process that area of the slide where, for example, onecolor of pen ink may indicate that a particular diagnostic evaluationmust be performed, while another color of ink may indicate that the areacontains a calibration or reference material and would call for thesystem to run a corresponding calibration procedure. Note that, inaddition to slides, the described technique may be readily adapted toexamine other mounting forms for microscopic material such as, forexample, microtiter plates or microarrays. Thus, it will be appreciatedby one skilled in the art that the capabilities of such embodiments ofthe system, configured to recognize particular dyes or inks, may extendto many different automatic scanning processes where interactive markingof areas of interest with specific pens, the pens having different colorinks previously evaluated by the system, may be used to automaticallydesignate and actuate a subsequent evaluation or other processing ofthat area of interest by an appropriate component of the system or otherspecified device.

Additionally, the artificial images of the field of view may alsofacilitate the presentation of the data in a configuration allowingidentification and selection of meaningful objects or areas of interestas, for example, still images in a report prepared for diagnostic orother reporting purposes. Still further, the artificial image of thefield of view may also be used to facilitate the identification andextraction of selected features of the treated sample. For example,marked point processes, contextual analysis, and/or geo-statistics maybe used to identify and extract features from the image based on, forinstance, a spatial distribution analysis of a particular dye. Such afeature extraction capability would also allow, for example, fields ofview or objects of interest to be sorted, flagged, or otherwiseidentified or grouped based on, for instance, the overall content of agiven marker dye or a selected ratio of particular marker. Where, forexample, a threshold criteria can be established, such a capabilitywould be the detection of rare, worsening, or other serious events.Proceeding further, classifiers based specifically on the imageprocessing resulting from the counterstain and/or marker dye specificimages may then be established and used to evaluate the presence ofcertain cell types or to perform a diagnosis based upon the field ofview. Such classifiers may usually also encompass other informativefeatures such as, for example, detail based upon the morphology or thetexture of the cells.

Still further, another advantageous aspect of the present invention isrealized where the system is capable of processing the image data at afaster rate than the images are acquired. The enhanced speed at whichthe image data is processed may allow, for example, features indicatedby a particular marker dye to be processed and classified. Accordingly,various conditions may be identified based upon predetermined criteria.As such, visual and/or sonic alarms may be established and/or mapped inconjunction with the processing of the image data. Thus, in someinstances, the operator's attention may be directed to a specific fieldof view or object of interest when a characteristic of a marker attainsa predetermined level in, for example, intensity or presence in aparticular field.

FIG. 12 is a schematic representation of a practical realization of anextended system configuration according to one embodiment of the presentinvention. In such an implementation, the system 100 or workstation iscentered about a microscope 150. The microscope 150 may include one ormore robotic components including, for example, a motorized stage, anautomatic focus mechanism, a motorized objective changer, and anautomatic light intensity adjustment. The system 100 may also includevarious input devices such as, for instances, cameras 300 a and 300 bhaving fast automatic focusing and configured for acquiringlow-resolution and high-resolution images, a flat bed linear scanners310 used for acquiring low-resolution images, a grossing station 320,and a voice-recording device 330, which are all linked to a computerdevice 350 through various data transmission links 400. The workstation100 can be part of a Local Area Network (LAN) 700, but may also beconfigured to support different communication protocols such thatavailable communication channels such as, for example, a standardtelephone line, an ISDN connection, or a T1 line, can readily connectthe workstation 100 to other components or devices over large distancesvia a Wide Area Network (WAN) 750 as will be appreciated by one skilledin the art.

If the pathology workstation 100 is configured to operate in anintegrated environment, the WAN 700 or LAN 750 connection may permitaccess to, for instance, existing reference databases 800 and HospitalInformation Systems (HIS) 850. With such a configuration, new samplesand/or cases may readily be compared with the pictures and accompanyinginformation of previously-accumulated reference cases. Further, imagesacquired from the samples and/or slides being examined at theworkstation 100 can be complemented with the patient and case history asnecessary.

In the extended configuration embodiment as shown in FIG. 12, thepathology workstation 100 is particularly configured for a comprehensivesample evaluation. For example, with information and digital pictures ofthe initial gross biological sample, images of the slides prepared fromthe sample can be prepared and processed as described herein. Thepatient and case information, the images, and the resulting quantitativeinformation about the cell components of the sample and the samplearchitecture (in the case of, for instance, tissue samples) can also becollected, integrated if necessary, and stored in a single database. If,for example, an initial or second expert opinion is needed or if theslide is used for training or proficiency testing, the communicationcapabilities of the extended configuration along with the automationfeatures of the microscope 150 may allow the workstation 100 to be usedas a tele-pathology system. For example, high-resolution images directedto features or objects of interest characterizing a questionablesituation on a particular slide may be electronically forwarded to theexpert and/or to the audited candidate. In some instances, an overviewpicture of the slide may be provided, wherein the automated microscope150 is used to scan the slide automatically on, for example, a field byfield basis. The corresponding digital images may then be stored in thememory of the computer device 350. Where a field by field basis is used,the edges of adjacent fields may be precisely matched using correlationalgorithms, so as to provide a single large overview image of the entireslide. Such an overview image may assist the reference pathologist inmaking an assessment of the information. In some instances, thereference pathologist may remotely control the workstation 100 from aremote site to acquire necessary and/or supplemental images which may berequired so as to provide a correct and thorough assessment of theslide.

Subsequently, the information accumulated by the workstation 100 for astudied case such as, for instance, real or mathematically generatedimages, measurement results and graphical representations thereof,patient data, preparation data, and screening maps, may be selectivelyintegrated into a report which can either be printed or accessedelectronically. Such a report would provide a comprehensive picture ofthe case under evaluation and would also facilitate quality assuranceand standardization issues.

It will be understood that the methodology and procedures detailedherein in conjunction with the system 100 specify a method ofquantifying an amount of a molecular species from an image of a samplecaptured by an RGB camera in a video-microscopy system. One skilled inthe art will also appreciate that such a method may be automated so asto provide a computer software program product, executable on a computerdevice, having executable portions capable of quantifying the amount ofa molecular species from a digital image of a sample captured by a colorimage acquisition device, such as an RGB camera, in a video-microscopysystem. Accordingly, embodiments of the system 100 describe theimplementation of the method and/or the corresponding computer softwareprogram product which may be accomplished in appropriately configuredhardware, software, or a combination of hardware and software inaccordance with the spirit and scope of the present invention.

Thus, embodiments of the present invention comprise a colorimetricanalysis technique for prepared samples that provides effectivedetection and quantification of species of interest that overcomeslimiting factors of prior art techniques such as, for example, spectraloverlapping, mixing of colors due to spatial overlap of membrane andnuclear markers, limited spectral resolution of the acquisition device,calibration particularities, the subjectivity of the detection andquantification process, and inconsistencies between human operators ofthe analysis equipment. Embodiments of the present invention furtherprovide an image processing technique which does not rely upon thesubjective detection of contrast within the prepared sample or a complexand voluminous analysis of the sample at specific wavelengths of lightusing a combination of light sources and filters. Therefore, embodimentsof the present invention provide a simpler and more effectivecolorimetric analysis technique that overcomes detection andquantification limitations in prior art analysis techniques, reducessubjectivity and inconsistency in the sample analysis, and is capable ofproviding the necessary analysis information about the sample, once animage of the sample is captured, without relying upon furtherexamination of the sample to complete the analysis.

More particularly and as demonstrated, the analysis (detection andquantification of a molecular species of interest) of the preparedsample is accomplished through the measurement of light intensities thatare manifested in a digital image of the sample captured by a colorimage acquisition device. Since the analysis is relativelyimage-dependent, rather than sample-dependent, redundant images may becaptured for analysis, while many samples may be processed so as tocapture the necessary images within a relatively short period of time.Once the image data has been captured and stored, the actual analysismay occur at a later time or as needed without requiring the physicalpresence of the actual sample. Such an analysis may be further appliedto examining the entire sample or even the entire slide. Thus,embodiments of the present invention provide an expeditious quantitativevideo-microscopy system that permits the use of such a system as aroutine or “production” tool capable of accomplishing a relatively highanalysis throughput. As such, significant advantages are realized byembodiments of the present invention as compared to prior artquantitative microscopy systems which were typically limited in samplethroughput and analysis, thus generally making such systems more usefulas research tools.

Many modifications and other embodiments of the invention will come tomind to one skilled in the art to which this invention pertains havingthe benefit of the teachings presented in the foregoing descriptions andthe associated drawings. Therefore, it is to be understood that theinvention is not to be limited to the specific embodiments disclosed andthat modifications and other embodiments are intended to be includedwithin the scope of the appended claims. Although specific terms areemployed herein, they are used in a generic and descriptive sense onlyand not for purposes of limitation.

1. A method of modeling a dye, said method comprising: determining atransmittance of the sample treated with the dye from a color image ofthe treated sample, the image comprising a plurality of pixels, in eachof a red, green, and blue channel of an RGB color space and for eachpixel of the image so as to form am RGB triplet for each pixel; groupingthe RGB triplets according to the minimum transmittance in the red,green, and blue channels for the respective RGB triplet; normalizingeach group RGB triplets by summing the transmittances in each of therespective red, green, and blue channels and then dividing each of thesummed transmittances by the number RGB triplets in the respective groupso as to form respective normalized RGB triplets; and tabulating thenormalized RGB triplets according to the minimum transmittance of eachnormalized RGB triplet so as to form a correspondence table for the dye,the correspondence table extending in transmittance increments between0% and 100% transmittance.
 2. A method according to claim 1 furthercomprising capturing an image of the treated sample with a color imageacquisition device so as to form the color image of the treated sample.3. A method according to claim 1 further comprising determining, when atransmittance increment in the correspondence table is without anormalized RGB triplet, an approximated transmittance in each of thered, green, and blue channels so as to form an approximated normalizedRGB triplet for that transmittance increment.
 4. A method according toclaim 3 wherein determining the approximated normalized RGB tripletfurther comprises determining a reference transmittance increment havinga normalized RGB triplet, both at a higher transmittance increment and alower transmittance increment in the correspondence table, with respectto the transmittance increment without the normalized RGB triplet, andthen interpolating between the respective transmittances in each of thered, green, and blue channels of the reference transmittance incrementsso as to form an approximated normalized RGB triplet for thetransmittance increment without the normalized RGB triplet.
 5. A methodaccording to claim 1 farther comprising establishing a significancethreshold for the number of RGB triplets in a group and, for any groupfailing to meet the significance threshold, discarding the RGB tripletstherein as being insignificant.
 6. A method according to claim 1 furthercomprising plotting the RGB triplets for the pixels of the image in anRGB color space so as to obtain a three-dimensional RGB representationof the respective dye, prior to normalizing each group of RGB triplets.7. A method according to claim 1 further comprising plotting thenormalized RGB triplets in an RGB color space so as to obtain acharacteristic RGB path of the respective dye through the RGB colorspace.
 8. A method according to claim 1 further comprising plotting thenormalized RGB triplets on a one-dimensional scale so as to graphicallyrepresent the correspondence table.
 9. A method according to claim 2wherein capturing an image of the treated sample further comprisescapturing an image of the sample in a video microscopy system with atleast one of an RGB camera and an RGB-configured scanner.
 10. A methodaccording to claim 2 wherein capturing an image of the treated samplefurther comprises illuminating the sample under Koehler illuminationconditions.
 11. A method according to claim 2 wherein capturing an imageof the treated sample further comprises capturing an image of the samplein a chromatic aberration-corrected video-microscopy system.
 12. Amethod according to claim 2 wherein capturing an image of the treatedsample further comprises illuminating the sample with a light source anddetermining a transmitted intensity of the light transmittedtherethrough in each of the red, green, and blue channels.
 13. A methodof modeling a combination of a plurality of dyes,said method comprising:forming a corresponding table for each of the plurality of dyes,comprising; determining a transmittance of a sample treated with therespective dye from a color image of the treated sample, the imagecomprising a plurality of pixels, in each of a red, green, and bluechannel of an RGB color space and for each pixel of the image so as toform an RGB triplet for each pixel; grouping the RGB triplets accordingto the minimum transmittance in the red, green, and blue channels forthe respective RGB triplet; normalizing each group of RGB triplets bysumming the transmittances in each of the respective red, green, andblue channels and then dividing each of the summed transmittances by thenumber of RGB triplets in the respective group so as to form respectivenormalized RGB triplets; and tabulating the normalized RGB tripletsaccording to the minimum transmittance of each normalized RGB triplet soas to form the correspondence table for the respective dye, thecorrespondence table extending in transmittance increments between 0%and 100% transmittance; and orthogonally adding the correspondencetables of the plurality of dyes so as to form a dye space representationof the plurality of dyes, the dye space representation having onedimension for each dye and providing a reference model for a combinationof the plurality of dyes.
 14. A method according to claim 13 furthercomprising capturing an image of the treated sample with a color imageacquisition device so as to form the color image of the treated sample.15. A method according to claim 13 further comprising determining, whena transmittance increment in the correspondence table for a dye iswithout a normalized RGB triplet, an approximated transmittance in eachof the red, green, and blue channels so as to form an approximatednormalized RGB triplet for that transmittance increment.
 16. A methodaccording to claim 15 wherein determining the approximated normalizedRGB triplet further comprises determining a reference transmittanceincrement having a normalized RGB triplet, both at a highertransmittance increment and a lower transmittance increment in thecorrespondence table for the dye, with respect to the transmittanceincrement without the normalized RGB triplet, and then interpolatingbetween respective transmittances in each of the red, green, and bluechannels of the reference transmittance increments so as to form anapproximated normalized RGB triplet for the transmittance incrementwithout the normalized RGB triplet.
 17. A method according to claim 13further comprising establishing a significance threshold for the numberof RGB triplets in a group and, for any group failing to meet thesignificance threshold, discarding the RGB triplets therein as beinginsignificant.
 18. A method according to claim 13 further comprisingplotting the RGB triplets for the pixels of the image in an RGB colorspace so as to obtain a three-dimensional RGB representation of therespective dye, prior to normalizing each group of RGB triplets.
 19. Amethod according to claim 13 further comprising plotting the RGBtriplets for the pixels of the images for each of the plurality of dyesin an RGB color space so as to obtain a three-dimensional representationof a combination of the plurality of dyes, prior to normalizing eachgroup of RGB triplets.
 20. A method according to claim 13 furthercomprising plotting the normalized RGB triplets in an RGB color space soas to obtain a characteristic RGB path of the respective dye through theRGB colorspace.
 21. A method according to claim 13 further comprisingplotting the normalized RGB triplets on a one-dimensional scale so as tographically represent the correspondence table.
 22. A method accordingto claim 13 wherein orthogonally adding the correspondence tables of theplurality of dyes further comprises orthogonally adding thecorrespondence tables of the plurality of dyes according to theLambert-Beer law.
 23. A method according to claim 13 further comprisingplotting the dye space representation of the plurality of dyes on ascale having a number of dimensions corresponding to the number of dyes.24. A method according to claim, 13 wherein orthogonally adding thecorrespondence tables of the plurality of dyes further comprisesorthogonally adding the correspondence tables of the plurality of dyesso as to form a resultant correspondence table for the combinedplurality of dyes the resultant correspondence table comprising aplurality of resultant RGB triplets extending between 0% and 100%transmittance.
 25. A method according to claim 14 wherein capturing animage of the treated sample further comprises capturing an image of thesample in a video microscopy system with at least one of an RGB cameraand an RGB-configured scanner.
 26. A method according to claim 14wherein capturing an image of the treated sample further comprisesilluminating the sample under Koehler illumination conditions.
 27. Amethod according to claim 14 wherein capturing an image of the treatedsample further comprises capturing an image of the sample in a chromaticaberration-corrected video-microscopy system.
 28. A method according toclaim 14 wherein capturing an image of the treated sample furthercomprises illuminating the sample with a light source and determining atransmitted intensity of the light transmitted therethrough in each ofthe red, green, and blue channels.
 29. A system for modeling a dyeindicative of a molecular species in a sample from an image of thesample treated with the dye, said system comprising: a computer devicecomprising: a processing portion configured to determine a transmittanceof the sample treated with the dye from a color image of the treatedsample, the image comprising a plurality of pixels, in each of a red,green, and blue channel of an RGB color space and for each pixel of theimage so as to form an RGB triplet for each pixel; a processing portionconfigured to group the RGB triplets according to the minimumtransmittance in the red, green, and blue channels for the respectiveRGB triplet; a processing portion configured to normalize each group ofRGB triplets by summing the transmittances in each of the respectivered, green, and blue channels and then dividing each of the summedtransmittances by the number of RGB triplets in the respective group soas to form respective normalized RGB triplets; and a processing portionconfigured to tabulate the normalized RGB triplets according to theminimum transmittance of each normalized RGB triplet so as to form acorrespondence table for the dye, the correspondence table extending intransmittance increments between 0% and 100% transmittance.
 30. A systemaccording to claim 29 further comprising a color image acquisitiondevice operably engaged with the computer system and configured so as tobe capable of capturing a magnified digital image of the sample, theimage acquisition device comprising at least one of an RGB-configuredscanner and a microscope operably engaged with an RGB camera.
 31. Asystem according to claim 30 wherein the computer device furthercomprises a processing portion configured to direct the imageacquisition device to capture the color image of the treated sample. 32.A system according to claim 29 wherein the computer device furthercomprises a processing portion configured to determine, when atransmittance increment in the correspondence table is without anormalized RGB triplet, an approximated transmittance in each of thered, green, and blue channels so as to form an approximated normalizedRGB triplet for that transmittance increment.
 33. A system according toclaim 32 wherein the processing portion for determining the approximatednormalized RGB triplet is further configured to determine a referencetransmittance increment having a normalized RGB triplet, both at ahigher transmittance increment and a lower transmittance increment inthe correspondence table, with respect to the transmittance incrementwithout the normalized RGB triplet, and then to interpolate between therespective transmittances in each of the red, green, and blue channelsof the reference transmittance increments so as to form an approximatednormalized RGB triplet for the transmittance increment without thenormalized RGB triplet.
 34. A system according to claim 29 wherein theprocessing portion for grouping the RGB triplets is further configuredto apply a significance threshold for the number of RGB triplets in agroup and, for any group failing to meet the significance threshold, todiscard the RGB triplets therein as being insignificant.
 35. A systemaccording to claim 29 wherein the computer device further comprises aprocessing portion configured to plot the RGB triplets for the pixels ofthe image in an RGB color space so as to provide a three-dimensional RGBrepresentation of the respective dye.
 36. A system according to claim 29wherein the computer device further comprises a processing portionconfigured to plot the normalized RGB triplets in an RGB color space soas to obtain a characteristic RGB path or the respective dye through theRGB color space.
 37. A system according to claim 29 wherein the computerdevice further comprises a processing portion configured to plot thenormalized RGB triplets on a one-dimensional scale so as to graphicallyrepresent the correspondence table.
 38. A system according to claim 30wherein the image acquisition device and the computer device are furtherconfigured to cooperate to form a chromatic aberration-correctedvideo-microscopy system.
 39. A system according to claim 30 furthercomprising a light source configured to illuminate the sample, whereinthe processing portion for determining a transmittance of the treatedsample is further configured to direct a measurement of a transmittedintensity of light transmitted through the sample in each of the red,green, and blue channels.
 40. A system according to claim 39 wherein thelight source is further configured to illuminate the sample underKoehler illumination conditions.
 41. A system for modeling a pluralityof dyes from a corresponding plurality of samples using an image of eachsample, each sample being treated with a different one of the pluralityof dyes, said system comprising: a computer device comprising: aprocessing portion configured to form a correspondence table for each ofthe plurality of dyes from the image of the respective samplecomprising; determining a transmittance of the sample treated with thedye from a color image of the respective treated sample, the imagecomprising a plurality of pixels, in each of a red, green, and bluechannel of an RGB color space and for each pixel of the image so ms toform an RGB triplet for each pixel; grouping the RGB triplets accordingto the minimum transmittance in the red, green, and blue channels forthe respective RGB triplet; normalizing each group of RGB triplets bysumming the transmittances in each of the respective red, green, andblue channels and then dividing each of the summed transmittances by thenumber of RGB triplets in the respective group so as to form respectivenormalized RGB triplets; and tabulating the normalized RGB tripletsaccording to the minimum transmittance of each normalized RGB triplet soas to form the correspondence table for the respective dye, thecorrespondence table extending in transmittance increments between 0%and 100% transmittance; and a processing portion configured toorthogonally add the correspondence tables of the plurality of dyes soas to form a dye space representation of the plurality of dyes, the dyespace representation having one dimension for each dye and providing areference model for a combination of the plurality of dyes.
 42. A systemaccording to claim 41 further comprising a color image acquisitiondevice operably engaged with the computer device and configured so as tobe capable of capturing a magnified digital image of each respectivesample, the image acquisition device comprising at least one of anRGB-configured scanner and a microscope operably engaged with an RGBcamera.
 43. A system according to claim 42 wherein the computer devicefurther comprises a processing portion configured to direct the imageacquisition device to capture the color image of the respective treatedsample.
 44. A system according to claim 41 wherein the computer devicefurther comprises a processing portion configured to determine, when atransmittance increment in the correspondence table is without anormalized RGB triplet, an approximated transmittance in each of thered, green, and blue channels so as to form, an approximated normalizedRGB triplet for that transmittance increment.
 45. A system according toclaim 44 wherein the processing portion for determining the approximatednormalized RGB triplet is further configured to determine a referencetransmittance increment having a normalized RGB triplet, both at ahigher transmittance increment and a lower transmittance increment inthe correspondence table, with respect to the transmittance incrementwithout the normalized RGB triplet, and then to interpolate between therespective transmittances in each of the red, green, and blue channelsof the reference transmittance increments so as to form an approximatednormalized RGB triplet for the transmittance increment without thenormalized RGB triplet.
 46. A system according to claim 41 wherein theprocessing portion for forming a correspondence table for each of theplurality of dyes is further configured to apply a significancethreshold for the number of RGB triplets in a group and, for any groupfailing to meet the significance threshold, to discard the RGB tripletstherein as being insignificant.
 47. A system according to claim 41wherein the computer device further comprises a processing portionconfigured to plot the RGB triplets for the pixels of the image in anRGB color space so as to provide a three-dimensional RGB representationof the respective dye.
 48. A system according to claim 41 wherein thecomputer device further comprises a processing portion configured toplot the RGB triplets for the pixels of the images for each of theplurality of dyes in an RGB color space so as to obtain athree-dimensional representation of a combination of the plurality ofdyes.
 49. A system according to claim 41 wherein the computer devicefurther comprises a processing portion configured to plot the normalizedRGB triplets in an RGB color space so as to obtain a characteristic RGBpath of the respective dye through the RGB color space.
 50. A systemaccording to claim 41 wherein the computer device further comprises aprocessing portion configured to plot the normalized RGB triplets on aone-dimensional scale so as to graphically represent the correspondencetable.
 51. A system according to claim 41 wherein the processing portionfor orthogonally adding the correspondence tables of plurality of dyesis further configured to orthogonally add the correspondence tables ofthe plurality of dyes according to the Lambert-Beer law.
 52. A systemaccording to claim 41 wherein the computer device further comprises aprocessing portion configured to plot the dye space representation ofthe plurality of dyes on a scale having a number of dimensionscorresponding to the number of dyes.
 53. A system according to claim 41wherein the processing portion for orthogonally adding thecorrespondence tables of the plurality of dyes is further configured toorthogonally add the correspondence tables of the plurality of dyes soas to form a resultant correspondence table for the combined pluralityof dyes the resultant correspondence table comprising a plurality ofresultant RGB triplets extending between 0% and 100% transmittance. 54.A system according to claim 42 wherein the image acquisition device andthe computer device are further configured to cooperate to form achromatic aberration-corrected video-microscopy system.
 55. A systemaccording to claim 41 wherein the processing portion for forming acorrespondence table for each of the plurality of dyes is furtherconfigured to direct a measurement of a transmitted intensity of lighttransmitted through the sample in each of the red, green, and bluechannels.
 56. A system according to claim 41 wherein the computer devicefurther comprises a processing portion configured to direct a lightsource to illuminate the sample under Koehler illumination conditions.57. A computer-readable medium encoded with a computer program capableof modeling a dye, indicative of a molecular species in a sample, froman image of the sample treated with the dye, said computer-readablemedium encoded with a computer program comprising: an executable portionconfigured for determining a transmittance of the sample treated withthe dye from a color image of the treated sample, the image comprising aplurality of pixels, in each of a red, green, and blue channel of an RGBcolor space and for each pixel of the image so as to form an RGB tripletfor each pixel; an executable portion configured for grouping the RGBtriplets according to the minimum transmittance in the red, green, andblue channels for the respective RGB triplet; an executable portionconfigured for normalizing each group of RGB triplets by summing thetransmittances in each of the respective red, green, and blue channelsand then dividing each of the summed transmittances by the number of RGBtriplets in the respective group so as to form respective normalized RGBtriplets; and an executable portion configured for tabulating thenormalized RGB triplets according to the minimum transmittance of eachnormalized RGB triplet so as to form a correspondence table for the dye,the correspondence table extending in transmittance increments between0% and 100% transmittance.
 58. A computer-readable medium encoded with acomputer program according to claim 57 further comprising an executableportion configured for directing a color image acquisition device tocapture a magnified digital image of a sample treated with the dye, theimage acquisition device comprising at least one of an RGB-configuredscanner and a microscope operably engaged with an RGB camera.
 59. Acomputer-readable medium encoded with a computer program according toclaim 57 further comprising an executable portion configured fordetermining, when a transmittance increment in the correspondence tableis without a normalized RGB triplet, an approximated transmittance ineach of the red, green, and blue channels so as to form an approximatednormalized RGB triplet for that transmittance increment.
 60. Acomputer-readable medium encoded with a computer program according toclaim 59 wherein the executable portion for determining the approximatednormalized RGB triplet is further configured for determining a referencetransmittance increment having a normalized RGB triplet, both at ahigher transmittance increment and a lower transmittance increment inthe correspondence table, with respect to the transmittance incrementwithout the normalized RGB triplet, and then interpolating between therespective transmittances in each of the red, green, and blue channelsof the reference transmittance increments so as to form an approximatednormalized RGB triplet for the transmittance increment without thenormalized RGB triplet.
 61. A computer-readable medium encoded with acomputer program according to claim 57 wherein the executable portionfor grouping the RGB triplets is further configured for applying asignificance threshold for the number of RGB triplets in a group and,for any group failing to meet the significance threshold, discarding theRGB triplets therein as being insignificant.
 62. A computer-readablemedium encoded with a computer program according to claim 57 furthercomprising an executable portion configured for plotting the RGBtriplets for the pixels of the image in an RGB color space so as toprovide a three-dimensional RGB representation of the respective dye.63. A computer-readable medium encoded with a computer program accordingto claim 57 further comprising an executable portion configured forplotting the normalized RGB triplets in an RGB color space so as toobtain a characteristic RGB path of the respective dye through the RGBcolor space.
 64. A computer-readable medium encoded with a computerprogram according to claim 57 further comprising an executable portionconfigured for plotting the normalized RGB triplets on a one-dimensionalscale so as to graphically represent the correspondence table.
 65. Acomputer-readable medium encoded with a computer program according toclaim 58 further comprising an executable portion configured fordirecting the image acquisition device and the computer device tocooperate to form a chromatic aberration-corrected video-microscopysystem.
 66. A computer-readable medium encoded with a computer programaccording to claim 57 wherein the executable portion for determining atransmittance of the treated sample is further configured for directinga measurement of a transmitted intensity of light transmitted throughthe sample in each of the red, green, and blue channels.
 67. Acomputer-readable medium encoded with a computer program according toclaim 57 further comprising an executable portion configured fordirecting a light source to illuminate the sample under Koehlerillumination conditions.
 68. A computer-readable medium encoded with acomputer program configured for modeling plurality of dyes from acorresponding plurality of samples using an image of each sample, eachsample being treated with a different one of the plurality of dyes, saidcomputer-readable medium encoded with a computer program comprising: anexecutable portion configured for forming a correspondence table foreach of the plurality or dyes from a magnified digital image of therespective sample captured by an image acquisition device, the imageacquisition device comprising at least one of an RGB-configured scannerand a microscope operably engaged with an RGB camera comprising;determining a transmittance of a sample treated with the respective dyefrom a color image of the treated sample, the image comprising aplurality of pixels, in each of a red, green, and blue channel of an RGBcolor space of and for each pixel of the image so as to form an RGBtriplet for each pixel; grouping the RGB triplets according to theminimum transmittance in the red, green, and blue channels for therespective RGB triplet; normalizing each group of RGB triplets bysumming the transmittances in each of the respective red, green, andblue channels and then dividing each of the summed transmittances by thenumber of RGB triplets in the respective group so as to form respectivenormalized RGB triplets; and tabulating the normalized RGB tripletsaccording to the minimum transmittance of each normalized RGB triplet soas to form the correspondence table for the respective dye, thecorrespondence table extending in transmittance increments between 0%and 100% transmittance; and an executable portion configured fororthogonally adding the correspondence tables of the plurality of dyesso as to form a dye space representation of the plurality of dyes, thedye space representation having one dimension for each dye and providinga reference model for a combination of the plurality of dyes.
 69. Acomputer-readable medium encoded with a computer program according toclaim 68 further comprising an executable portion configured fordirecting a color image acquisition device to capturing a magnifieddigital image of each respective sample, the image acquisition devicecomprising at least one of an RGB-configured scanner and a microscopeoperably engaged with an RGB camera.
 70. A computer-readable mediumencoded with a computer program according to claim 68 further comprisingan executable portion configured for determining, when a transmittanceincrement in the correspondence table is without a normalized RGBtriplet, an approximated transmittance in each of the red, green, andblue channels so as to form an approximated normalized RGB triplet forthat transmittance increment.
 71. A computer-readable medium encodedwith a computer program according to claim 70 wherein the executableportion for determining the approximated normalized RGB triplet isfurther configured for determining a reference transmittance incrementhaving a normalized RGB triplet, both at a higher transmittanceincrement and a lower transmittance increment in the correspondencetable, with respect to the transmittance increment without thenormalized RGB triplet, and then interpolating between the respectivetransmittances in each of the red, green, and blue channels of thereference transmittance increments so as to form an approximatednormalized RGB triplet for the transmittance increment without thenormalized RGB triplet.
 72. A computer-readable medium encoded with acomputer program according to claim 68 wherein the executable portionfor forming a correspondence table for each of the plurality of dyes isfurther configured for applying a significance threshold for the numberof RGB triplets in a group and, for any group failing to meet thesignificance threshold, to discard the RGB triplets therein as beinginsignificant.
 73. A computer-readable medium encoded with a computerprogram according to claim 68 further comprising an executable portionconfigured for plotting the RGB triplets for the pixels of the image inan RGB color space so as to provide a three-dimensional RGBrepresentation of the respective dye.
 74. A computer-readable mediumencoded with a computer program according to claim 68 further comprisingan executable portion configured for plotting the RGB triplets for thepixels of the images for each of the plurality of dyes in an RGB colorspace so as to obtain a three-dimensional representation of acombination of the plurality of dyes.
 75. A computer-readable mediumencoded with a computer program according to claim 68 further comprisingan executable portion configured for plotting the normalized RGBtriplets in an RGB color space so as to obtain a characteristic RGB pathof the respective dye through the RGB color space.
 76. Acomputer-readable medium encoded with a computer program according toclaim 68 further comprising an executable portion configured forplotting the normalized RGB triplets on a one-dimensional scale so as tographically represent the correspondence table.
 77. A computer-readablemedium encoded with a computer program according to claim 68 wherein theexecutable portion for orthogonally adding the correspondence tables ofthe plurality of dyes is further configured for orthogonally adding thecorrespondence tables of the plurality of dyes according to theLambert-Beer law.
 78. A computer-readable medium encoded with a computerprogram according to claim 68 further comprising an executable portionconfigured for plotting the dye space representation or the plurality ofdyes on a scale having a number of dimensions corresponding to thenumber of dyes.
 79. A computer-readable medium encoded with a computerprogram according to claim 68 wherein the executable portion fororthogonally adding the correspondence tables of the plurality of dyesis further configured for orthogonally adding the correspondence tablesof the plurality of dyes so as to form a resultant correspondence tablefor the combined plurality of dyes the resultant correspondence tablecomprising a plurality of resultant RGB triplets extending between 0%and 100% transmittance.
 80. A computer-readable medium encoded with acomputer program according to claim 69 further comprising an executableportion configured to direct the image acquisition device and thecomputer device to cooperate to form a chromatic aberration-correctedvideo-microscopy system.
 81. A computer-readable medium encoded with acomputer program according to claim 68 wherein the executable portionfor forming a correspondence table for each of the plurality of dyes isfurther configured for directing a measurement of a transmittedintensity of light transmitted through the sample in each of the red,green, and blue channels.
 82. A computer-readable encoded a computerprogram according to claim 68 further comprising an executable portionconfigured for directing a light source to illuminate the sample underKoehler illumination conditions.